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TIME SHIFTING BY SIMULTANEOUSLY RECORDING 
AND PLAYING A DATA STREAM 

Cross-Ref erence to Related Application 

This is a continuation-in-part of U.S. patent 
application, serial no. 09/150,577, filed September 10, 
1998, which is a continuation-in-part of U.S. Patent 
Application Serial No. 08/996,535, filed December 25, 1997. 

Background 

This present invention relates to the recording and 
playing back of a video stream. A video stream includes 
any combination of audio and/or video data streams. 

Video streams have typically been recorded on analog 
media such as a video cassette. A video cassette recorder 
(VCR) is used to record the video stream on the video 
cassette. The video stream may come via a broadcast 
signal, via cable, via satellite signal, or from another 
video playback device. Once the video stream has been 
recorded, the VCR is used no rewind the recording medium 
and play what was recorded. However, due to the nature of 
the analog medium, once the VCR has started recording, it 
is not possible to play back the portion of the video 
stream that has already been recorded until the recording 
session is terminated. 

Thus, there is 'a need for an improved way of recording 
portions of a video transmission. 



Summary 

In accordance with one embodiment a method of 
displaying and recording a video transmission includes 
monitoring an ongoing video transmission for a 
5 characteristic. Upon detecting the characteristic, the 
video transmission is automatically recorded while the 
video transmission is being displayed. 

Description of the Drawings 
10 Figure 1 is a block diagram of a video record and 

playback system in accordance with one embodiment of the 
invention; 

Figure 2 is a flow chart of one embodiment of the 
method of providing a time-shifted video stream in 
15 accordance with the invention; 

Figure 3 shows one embodiment of performing block 206 
of Figure 2; 

Figure 4 shows one embodiment of an apparatus for 
storing the video stream on a hard disk in accordance with 
2 0 the invention; 

Figure 5 shows an embodiment of a method of using the 
storage unit as a temporary buffer in accordance with the 
invention . 

Figure 6 is a flowchart of the playback of a video 
25 stream catching up to the incoming video stream for one 
embodiment of the invention; 

Figure 7 shows another embodiment, in which the user 
is able to suspend the display of the incoming video stream 
in accordance with the invention; 
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Figure 3 shows an example of an embodiment of a 
display screen with several image frames taken at different 
times in accordance with the invention; 

Figure 9 is a flow chart showing one embodiment of a 
system for recording and playing back a video stream in 
accordance with the invention; 

Figure 10 is a block diagram of a computer system 
useful in implementing one embodiment of the present 
invention; 

Figure 11 is an exemplary TV display for implementing 
an embodiment of the present invention; 

Figure 12 is a flow chart showing the operation of 
anon her embodiment of the present invention; 

Figure 12A is a continuation of the flow chart of 
Figure 12 . 

Figure 13 is a top plan view of a remote control 
useful in one embodiment of the present invention; 

Fiaure 14 is a block diagram showing how data is 
transferred to and from a memory device in accordance with 
one embodiment of the present invention; 

Figure 15 is a flow chart showing the flow of input 
video information into a storage device in accordance with 
one embodiment of the present invention; 

Figure 16 is a flow chart showing the flow of data 
from the storage device in accordance with one embodiment 
of the present invention; 

Figure 17 shows a display with an electronic program 
guide in accordance with one embodiment of the present 
invention; 



Figure 18 is a block diagram of a video transmission 
system in accordance with one embodiment of the present 
invention; 

Figure 19 is a depiction of a screen display in 
5 accordance with the embodiment shown in Figure 13; 

Figure 20 is a flow chart for one embodiment of 
software for implementing the system shown in Figure 18 in 
accordance with the invention; and 

Figure 21 is a flow chart for one embodiment of 
10 software for implementing another aspect of the system 
shown in Figure 18 in accordance with the invention. 

Detailed Description 
Figure 1 shows a block diagram of a video record and 

15 playback system 100 in accordance with one embodiment of 
the invention. A video stream is received at the video 
input port 102. The video stream may be provided by a 
camera, a television signal, broadcast, cable, or satellite 
signals, or another video playback device. In one 

2 0 embodiment, an analog- to -digital conversion may be 

performed on an analog video stream to form a digital video 
bit stream. In a different embodiment, the video is 
already in digital form. The video record and playback 
system 100 may be part of a system, such as a computer 

25 system or set top box, such that the video input port 102 
may be part of a video capture card in the computer system 
or set top box. 

The digital video stream from the video input port 102 
is optionally compressed at compression unit 104. In one 

30 embodiment, the video is already compressed, such as would 
be the case with an Moving Picture Experts Group 2 (MPEG 2) 
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specification (available from the International Standards 
Organization Standard IS01172) compliant video signal, and 
no further compression is needed. The video stream is then 
stored in the storage unit 108. A buffer unit 106 may be 
used as temporary storage for providing larger sequential 
blocks of video data to the storage unit 108. In one 
embodiment, the buffer unit 106 comprises a random access 
memory that allows relatively quick access tc any portion 
of the stored video stream. 

The video stream is played back by reading the video 
stream from the storage unit 108. If the video stream was 
compressed in compression unit 104, then a decompression 
unit 110 decompresses the retrieved video stream. The 
video stream is provided from a video output port 120, to a 
monitor or ether display device such as a TV to provide 
sound and/or video to a user. 

A removable storage unit 122 may also be included in 
video record and playback system 100, Examples of 
removable storage units include a writeabie compact disk 
read only memory (CD-ROM) , writeabie digital video disk 
(DVD), a flash memory, or another hard disk. The 
availability of a removable storage unit 122 allows a user 
to transfer a recording of a video stream stored in storage 
unit 108 to the removable storage unit 122 and then to 
transfer the unit 122 to another system at a different 
location. 

In one embodiment, a processor 130 controls the 
operations of the video record and playback system 100. 
The compression unit 104 and decompression unit 110 may be 
implemented in hardware, or the compression and 
decompression functions of units 104 and 110 may be 



performed by the processor 130. Processor 130 receives 
instructions from firmware /memory 140, using technology 
that is well known. 

Figure 2 shows a flow chart of an embodiment of a 
method of providing a time-shifted video stream in 
accordance with the present invention. The flow chare 
begins at block 200, and continues at block 202 where the 
video stream is received. The recording of the video 
stream begins at block 204. At block 206, playback of the 
recorded video stream is performed by retrieving a portion 
of the video stream from the random access storage unit 
while the recording of the incoming video stream continues. 
The retrieved portion of the video stream may be time- 
shifted from the incoming video stream by a time delay. At 
block 203, the portion of the video stream retrieved from 
the random access storage unit is retrieved for display by 
a television or other display device. 

In this way, the record and playback functions are 
decoupled from one another. The user may now begin 
watching a recorded TV shew from the beginning, e.g., prior 
to the show being completely recorded. 

Figure 3 shows an embodiment of a method for 
retrieving a portion of the video stream from the random 
access storage unit while continuing to record the incoming 
video stream in accordance with the invention, although the 
invention is not restricted to this embodiment. In the 
disclosed embodiment, the substantially simultaneous 
recording and playback of the video stream is performed by 
multiplexing or alternately storing the video stream to the 
random access storage unit and reading cf the video stream 
from the random access storage unit. The multiplexed or 



alternated stores and reads may occur quickly enough that 
the user dees not notice an appreciable delay in the 
playback of the video stream, and the incoming video stream 
is not lost, e.g., all of the video stream is recorded. 
5 Thus, the record and playback are substantially 
simultaneous from the user's point of view. 

In one embodiment, the random access storage unit 
comprises a hard disk. The recrieval of the time -shifted 
video signal from the hard disk is performed at a firsc 
10 physical location (or sector) of the hard disk, and the 
storing to the hard disk of the incoming video stream is 
performed at a different physical location (or sector) on 
the hard disk. Because it takes more time to jump back and 
forth between different sectors of the hard disk than :o 
15 read and write to sequential locations in the same sector, 
data may be buffered to reduce the number of accesses co 
and from the hard disk using buffer 106. This increases 
the amount of data transferred per access. 

Additionally, because of time constraints for reading 
20 and writing to the hard disk, data may be compressed and 

decompressed to speed transfers to and from the hard disk. 

In block 302, the video stream is stored in a random 
access storage unit. The video stream is optionally 
compressed and/or buffered prior to storage. In block 304, 
25 the video stream is retrieved from the random access 
storage unit. Buffering and/or decompression may be 
performed before providing the retrieved video stream to 
the video output port 120. 

The next portion of the video stream is scored (block 
30 306) as described in block 302. At block 308, the next 
portion of .the video stream is retrieved as described in 
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block 304. This process is repeated until either the 
recording or playback cycle is terminated. 

Figure 4 shows one embodiment of an apparatus for 
storing the video stream in the storage unit 103 in 
accordance with the invention. Again, the invention is not 
restricted in scope to the illustrated embodiments. In 
this embodiment, the video stream is stored as separate 
files 001 and 009 on a hard disk, for example. The 
processor 130 keeps track of the file and offset into the 
file of the data being played back, as well as the file and 
offset into the file of the data being recorded. If the 
random access storage unit is fast enough, more chan one 
video stream may be recorded and played back at the same 
time . 

Due to the nature of the random access storage unit 
being capable of easily recording ' over itself, the random 
access storage unit may act as a temporary buffer for 
recording the latest portion, or X number of minutes, of an 
incoming video stream, where X is set up based upon the 
size of the storage unit in this particular embodiment:. In 
one embodiment, X could be set up to be the entire storage 
unit. As newer portions of the video stream are received, 
they overwrite the older portions of the video stream saved 
in the random access storage unit. In this manner, the 
temporary buffering of the video stream acts as a circular 
buffer. In one embodiment, the processor 130 maintains 
pointers to the beginning and ending points of the 
temporary buffer. The processor 130 reassigns the pointers 
as newer portions of the video stream are received and/or 
older portions of the video stream are overwritten. 
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Figure 5 shows a flow chart 500 of an embodiment of 
one method for using a temporary buffer in accordance wich 
the invention. At block 502, the video stream is received 
at an input. Recording of the video stream to the storage 
unit begins at block 504. At block 506, older portions of 
the video stream are deleted as newer portions of the video 
stream are stored to the storage unit. 

A user may initiate a playback cycle following block 
506. For example, this may occur when the user wishes to 
re-view a video clip that he just saw. In one embodiment, 
the user stops recording to the temporary buffer and plays 
back the last portion of the temporary buffer. 

However, it may be more desirable to the user to be 
able to continue recording as shown at block 508. A record 
and playback cycle (as described with respect to Figure 2) 
is started, in which the incoming video stream is recorded 
while the user re-views the last portion of the temporary 
buffer. In this manner, after re-viewing the desired video 
clip, the user can resume sequentially watching the video 
stream from the current point of the incoming video for 
substantially simultaneous playback and record. 

At block 510, after the record and playback cycle is 
completed, all or part of the temporary buffer may be 
saved. Since the temporary buffer stores the latest X 
minutes of the video stream prior to the record and 
playback cycle, all or part of the temporary buffer may be 
allocated to the portion of the video stream saved during 
the record and playback cycle. Other portions of the video 
stream may then be deleted from the storage unit, or they 
may be marked as overwri teable and used as a new temporary 



Figure S illustrates an embodiment of a method for 
playing back a video stream to allow the playback to catch 
up to the incoming video stream in accordance with the 
invention. If the playback is performed at an overall rate 
faster than the rate at which the incoming video stream is 
received, then the playback may catch up to the incoming 
video stream. 

For example, playback of the video stream may have an 
overall rate faster than the rate of the incoming video 
stream if the playback is fast forwarded, or if segments of 
the playback are skipped altogether. When the time delay 
of the time-shifted video stream being played back fails 
below a certain threshold, the video and playback syscem 
100 will cease providing the time-shifted video stream from 
the storage unit. Instead, the incoming video stream will 
be provided to the video output port 120 directly. In one 
embodiment, a bypass 142, as shown in Figure 1, allows the 
incoming video stream to be provided to the video output 
port 120 directly. 

When this happens, the user has caught up to the 
"live" broadcast, e.g., the incoming video scream in this 
embodiment. The user may terminate the recording cycle, if 
he wishes. Alternatively, the user may put che video 
record and playback system 100 back into the temporary 
buffering mode in which only the latest portion of the 
video stream is stored. 

Figure 7 shows an embodiment in which the user is able 
to suspend the display of the incoming video stream. This 
can be used for example, when the user is interrupted, and 
wishes to continue viewing the video stream after the 
interruot ion . When interrupted, the user signals to the 



video stream and playback system 100 to suspend the display 
of incoming video stream. This can be done via a remote 
control, for example. At block 702, the video output 
signal is suspended. In one embodiment, the video output 
port continues to provide a still image of the image 
present at the instance when the suspend was encountered. 

At block 704 the incoming video stream is recorded but 
is not displayed to the monitor. Instead the playback is 
paused at the point at which the user indicated that the 
incoming video stream be suspended. When the user is ready 
to view the video stream again, he can signal the video 
record and playback system 100 to un- suspend the video 
stream so that it plays back from the point at which the 
video stream was suspended, as shown in block 706. 

The user may then view the video stream time shifted 
by the amount of time that he suspended the incoming video 
stream, or he may fast forward (or rewind) through the 
time-shifted video stream. When playback of the time- 
shifted video stream catches up to the point at which the 
incoming video stream is being recorded, the record and 
playback system 100 may display the incoming video stream 
directly from incoming video stream without retrieving the 
video stream from the storage unit, as described with 
respect to Figure 6. The recording of the video stream may 
then be terminated by the user, if desired. 

Employing a random access storage unit for storage of 
the video stream facilitates jumping to various points 
within the video stream. One way of jumping is by 
retrieving different time slices of the video stream. For 
example, an image frame from the video stream can be 
retrieved from the storage unit at 1 minute intervals from 
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a current position of the video stream. In one embodiment , 
an image frame at the current playback position + 1 minute, 
current playback position + 2 minutes, and so forth are 
retrieved and displayed cn the TV screen in a mosaic 
5 pattern. 

Figure 3 shows an example of a display screen 30 0 
which displays several image frames taken from the video 
stream at different times. In Figure 3, the current 
playback position is designated as X. In one embodiment:, 

10 the time interval, T, is user programmable. The intervals 
may be multiples of the time interval, as shewn. A small 
interval may be used if the user wishes to skip a 
commercial, which usually lasts only a few minutes. Longer 
intervals such as a half hour may be useful for determining 

15 which movies are recorded on a storage unit. 

After the image frames are displayed, the user is able 
to select one of the frames as a new starting point to 
which to begin an operation, such as a playback or record 
operation. 3y using such a story bearding method, it is 

20 easy for a user to quickly jump to a desired location 
within the video stream. 

In one embodiment, the record and playback system 100 
is able to detect a black screen or fade out, such as those 
which accompany the beginning or end of a commercial. This 

25 is useful in editing a video stream. The system may also 
record a series of cue points (e.g., fade outs and fade to 
black) for later reference. 

Having the video stream stored on a random access 
storage unit such as a hard disk allows for easy editing of 

30 the video stream. Individual frames of the video stream 
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may be deleted or replaced. For example, a commercial may 
be replaced by a fade- to-black sequence. 

Once the video stream on the storage unio has been 
edited, it can be stored to a more permanent r.edium, such 
as a writeable CD-ROM, flash memory, or another hard disk 
via the removable storage unit 122. 

Referring now to the embodiment shown in Figure 9, a 
flow chart 900 for digitally recording a video stream in 
accordance with an embodiment of the invention begins by 
capturing the video stream as indicated in block 902. If 
the stream is an analog stream, it may be digitized in an 
analog- to-digital conversion process as indicated at block 
904. Next the digital stream may be encoded and 
compressed, for example using ohe MPEG2 compression scheme, 
as indicated in block 906. The stream is alternately read, 
as indicated at block 910, and stored, as indicated in 
block 908, in a conventional storage device such as a hard 
disk drive, a digital video disk or a flash memory. Data 
that is read from the storage device may be decoded and 
decompressed using conventional technology, for example, as 
indicated in block 912, for display as indicated in block 
914 . 

A system 1000 in accordance with one embodiment of the 
present invention, shown in Figure 10, includes a processor 
1002. In one embodiment, the processor may be coupled to 
an accelerated graphics port (AGP) chipset: 1004 for 
implementing an accelerated graphics port embodiment. The 
chipset 1004 communicates with the AG? port 1005 and the 
graphics accelerator 1006. The television 1010 may be 
coupled to the video output of the graphics accelerator 
1006. The chipset 1004 accommodates the system memory 



1008. The chipset 1004 is also coupled to a bus 1012 which 
may be, for example, a peripheral component interconnect 
(PCI) bus (PCI Local Bus Specification, Version 2.1 June 1, 
1995) . The bus 1012 couples to TV tuner/capture card 1014 
5 which is coupled to an antenna 1015 or other video input 
port, such as a cable input pert, a satellite 
receiver/antenna or the like. The TV tuner and capture 
card 1014 selects a desired television channel and also 
performs the video capture function (block 9C2, Figure 9). 

10 One example of video capture card is the ISVR-III video 
capture card available from Intel Corporation. 

The bus 1012 is also coupled to a bridge 1016 which 
couples a hard disk drive 1013. The software 1020, 1022, 
1024, 1026, 1023, and 1030 may be stored on the hard disk 

15 1013. The bridge 1016 is also coupled to another bus 1032. 
The bus 1032 may be coupled to a serial input/output (SIO) 
device 1034. The device 1034 is in turn coupled to 
conventional components such as a mouse 1036, a keyboard 
1033, and an infrared interface 1040. Also coupled to the 

20 bus 1032 is a basic input/output system (BIOS; 1042. 

An example of a display 1100, shown in Figure 11, for 
the television 1010 may display a conventional television 
picture or program 1101 and, in addition, may have 
superimposed over the screen, in a discrete fashion, a 

25 mouse selectable series of icons, such as the icons 1102 
through 1114. When the mouse cursor 1116 selects the 
appropriate one of the icons, a corresponding feature may 
be implemented. Among the features that may be implemented 
in this fashion include a pause or stop function 1102, a 

30 start record function 1104, a fast forward function 1106, a 
rewind function 1103, and a 10 second replay function I 110 



which winds back 10 seconds (or some other amounc) and 
replays, and a catchup function 1112 that begins playing 
back recorded content at a faster than normal ra:e until 
the display catches up with a live broadcast and an 
electronic program guide 1114. The above described 
functions 1102, 1104, 1106, 1108, 1110, and 1112 may also 
be individual buttons on a remote control unit. 

Referring next to Figure 12, a program 102 0 called 
display is used to control various features implemented by 
the computer 1000. Initially a check is made at diamond 
1200 to determine if an electronic programming guide (E?G) 
is implemented. An electronic programming guide is an 
electronic depiction of the various programs that are 
available at different times. The electronic programming 
guide can be provided on a disk, over a modem, by an 
Internet connection and using an interactive broadcasting 
transmission such as Intercast" 2.0 interactive broadcasting 
software available from Intel Corporation although the 
invention is not limited in scope in this respect. 

If a particular television program is selected (even 
if the program is only selected for viewing) on the E?G, an 
identifier for that particular program may be stored, in 
one embodiment, as indicated at block 1202, and automatic 
recording of the program begins (block 1204) . In this way 
the stored version of the program is available to implement 
the various features described herein even if the storage 
function was not particularly requested. Because the 
information is stored in an overwriting fashion, no harm is 
done by recording the information even if the information 
turns out not to be useful. 



Referring to Figure 17, an electronic program guide 
user interface 1700 may be deployed on the television 1010. 
When the user mouse clicks a box image 1702, representing a 
given television program, using the cursor 1116, than 
program is automatically recorded. 

Continuing in Figure 12, an inquiry (diamond 1206) 
checks whether the user has selected the zcom feature for 
playback. If the user presses a zoom button during 
playback, a zoom feature is implemented in one embodiment. 
In one embodiment, five predefined quadrants in the 
television display may be defined including four quadrants 
located peripherally around a central quadrant:. When the 
user selects the zoom feature the user indicates the 
appropriate quadrant for the zcom (block 1203} . The 
selected quadrant is scaled (block 1210) to produce a 
larger (zoom) picture. 

Referring to block 1208, the pointer focus is 
identified to determine which quadrant will be zoomed. 
That quadrant is then scaled as indicated ac block 1210. 
Because the television program is continually being stored, 
the scaling can be implemented without any loss of 
continuity of the television picture. After the user has 
completed looking at the zoomed depiction, the user can 
return to the program at the point where he or she departed 
to view the zoomed depiction. Thereafter, the displayed 
depiction viewed by the user may be time delayed from the 
actually broadcast program. However the user is able to 
view the program in its entirety even if slightly time 
delayed. 

Next, a query is made at diamond 1212 to determine 
whether a pause function has been selected. If so, the 



playback is stopped as indicated at block 1214 but 
recording continues unabated as shown in block 1216. 

The next check determines whether the play feature has 
been selected (block 1218) . If so the recorded content: is 
5 played thereby terminating the paused display. However, 
while the recorded content is displayed, recording 
continues as indicated in block 1222 so that the recorded 
content may be eventually displayed without disrupting the 
continuity of the program. 
10 A query finds out if the fast forward feature has been 

implemented as indicated in diamond 1224. If so, a fast 
playback is implemented as shown in block 1225. Once the 
playback equals the current broadcast or program content 
(diamond 1223) , the flow returns to display current 
15 broadcast or program content. The fast forward may 

progress at 300 to 400% (or some other speed) of normal 
playback speed. 

Moving on to Figure 12A, at diamond 123 0 a check is 
done to see if the 10 second replay feature has been 
20 activated. If so, the storage device is operated to find 

the content which was recorded 10 seconds ago (block 1232} . 
The recorded content is then played (block 1234) from the 
10 second period back to present. This would correspond to 
the replay feature implemented with broadcast television 
25 sporting events. After the "rewind" and playback of 10 
seconds has been completed, the system is automatically 
transferred to automatic play at the content that was 
recorded while the replay is being displayed so the user 
does not lose any continuity in the show. Of course, the 
30 user can set the amount of time for the replay to be 
something other than 10 seconds. 



17 



A catchup feature is checked (diamond 1236) and, if 
this feature has been selected, an accelerated playback 
option is implemented (as indicated in block 1233) . In the 
accelerated playback, the playback may be slightly 
increased in speed, for example from 105 to 115% of actual 
speed, so the user can easily follow the program but can 
make up for lost time to get back into the program as 
broadcast. Once the recorded program catches up to the 
real-time program as indicated in diamond 124 0, the catchup 
feature is completed and the user returns to real-time 
display. A conventional technique may be used to maintain 
audio pitch during accelerated playback. 

Control over the record and playback functions may be 
implemented through a normal mouse function, for example, 
by clicking on icons as indicated in Figure 11. In 
addition, the record and playback features may be 
controlled remotely even when the user is net proximate to 
a keyboard or mouse. This may be done using a conventional 
remote control operator which may, for example, utilize 
infrared radiation . 

As indicated in the depiction of an example of a 
system 1000 (Figure 10) , an infrared adapter 1040 may be 
provided, for example, in compliance with the standards and 
specifications (such as Infrared Data Association Serial 
Infrared Link Access Protocol Version 1.0, June 23, 1994) 
of the Infrared Data Association (which can be found at 
{www.irda.org}). The remote control 1300, as shown in 
Figure 13, may be utilized as a separate stand alone remote 
control or its features may be incorporated into a global 
remote control. The illustrated control 1300 is a stand 
alone control for controlling the record and broadcast 



features. It includes an infrared transmission port 1302 
and a plurality of buttons to control the various features 
For example, a zoom function 1312 may be implemented as a 
four-way rocker switch. A zoom is implemented for the top 
quadrant of a display by pressing the upper edge of the 
button. If the button is pressed in the middle, the cente 
quadrant is chosen for zoom. Likewise any of the remainin 
three edges of the zoom button can be operated to select a 
desired quadrant for zoom features. 

A button 13 06 is provided to implement the replay 
function, a button 1308 may implement the pause feature, 
and an enter button 1310 may be provided to allow encry of 
various information including a particular television 
program station. An on/off button 1314 may also be 
provided. Fast forward may be implemented by buceon 1315, 
an electronic program guide may be called by pressing 
button 1304. The catch up feature may be called by the 
button 1313, and a 10 second reply may be implemented by 
the button 1320. A conventional numeric keyboard 1322 may 
be provided to input additional information including 
selected television channels. 

Techniques for storing the video stream onto the 
storage device 1410 and for reading the information out of 
the storage device are summarized in Table 1 below and 
illustrated in Figures 14, 15, and 16. In Figure 14, a 
schematic depiction of the storage system 1400 includes a 
digital storage device such as a hard disk drive 1410. Th 
digitized video 1402 is initially stored in a buffer which 
is designated as being currently on the top of the memory 
stack. The transfer of information between the buffers an 
the storage device 1410 may be dene in discrete time ■• 
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5 periods referred to herein as time steps. In a first time 
step, shown in Figure 15, the digitized video 1402 (Figure 
14) is stored in memory buffer 1404 because that buffer is 
currently at the top of the memory stack, as indicated in 
block 1502 in Figure 15. 

10 As the buffer 1404 fills up, the buffer 1403 mpves to 

the top of the stack (as indicated by the dashed arrow) and 
incoming video is stored in buffer 1408. As indicated in 
block 1504 in Figure 15 in time step 2, the buffer 1408 
replaces the buffer 1404 as the designated top of the stack 

15 buffer. The next input video is then stored in the new 
buffer (1408) as indicated in block 1506. 

In time step 3 the buffer 1408 has filled up and the 
contents of buffers 1404 and 1408 are written to the 
storage device 1410 in a single write operation. During 

20 the same time step, buffer 1406 moves to the top of the 
stack and becomes the storage buffer for incoming video. 
This is illustrated in blocks 1508, 1510 and 1512 in Figure 
15. 

In time step 4, the buffer 1404 moves back to the top 
25 of the stack to store incoming video since its previous 

content has already been saved in the storage device 1410. 
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This is indicated in block 1514 of Figure 15. 
Additionally, in time step 4, the content of buffer 1403 is 
written to the storage device 1410 as illustrated in block 
1515. The storing of incoming information, as illustrated 
in Figure 15 then cycles back to the top of the flow in 
Figure 15 and continues in the same fashion thereafter. 

The content from the storage device 1410 is then read 
into buffers 1412 and 1414 in time step 4 and block 1602. 

In time step 5, the buffer 1408 moves to the top of 
the stack to store incoming video, and buffer 1414 moves to 
the top of the output stack and transfers data to the video 
output 1413. The contents of the buffers 1404 and 1405 are 
then written to the storage device 1410. 

The time steps 1 to 5 illustrate a complete cycle from 
input to output. The remaining sequence of steps repeat 
starting at step 1 through 4 for the input of data and time 
steps 4 through 9 repeat for the output of data. 

Referring now solely to Figures 14 and 16, in time 
step 6, information stored in the storage device is read to 
the sixth and fourth buffers (i.e., buffers 1414 and 1416) 
as indicated in block 1506. The contents of the fifth 
buffer (1412) are sent to the output port 1413. 

In time step 7, the contents of the sixth buffer 
(which is buffer 1416) are sent to the output port 1418. 
No other output operations occur. 

In time step 8, the contents from the storage device 
1410 are read into the fifth and sixth buffers (buffers 
1412 and 1416) as indicated in block 1612. Also the 
contents of the fourth buffer (buffer 1414) are passed to 
the output port 1413 as indicated in block 1614. 



Finally, during time step 9 the contents cf the fifth 
buffer (buffer 1412) are passed to the output port 1413, 
completing the cycle. 

Using these techniques, the storage device is provided 
with enough time, through the operation of the buffers, to 
write the incoming video while supplying enough data 
simultaneously to the output display. Since the memory is 
used to make sure no content is dropped at the input end 
and the display is never starved at the output end, 
continuous display can be provided without losing 
information. In addition reading and writing larger chunks 
of data at one time reduces the amount cf storage device 
head movement, thereby allowing the storage device to keep 
up with the read and write requests. 

The size of the individual buffers 1404 zo 1403 and 
1412 to 1416 ("B") in megabytes is dependent a: least in 
part on a number of factors including the inpuc/output data 
rate n D" , for example in megabytes per second, the 
effective bandwidth of the storage device when reading or 
writing W S" , for example in megabytes per second, and the 
average seek time for the storage device per transaction 
"t" , in microseconds. The time that it takes to fill up 
one memory buffer (B divided by D) is advantageously 
greater than the time it takes to read or write two memory 
buffers (2B divided by S) plus the average seek time (t) : 




Solving for the minimum buffer size in the above 
equation yields the following equation which gives the 
minimum buffer size: 



[S-2DJ UOOOOOO 



A viewer can pause ,a season premier of a television 
while the station is still broadcasting in order to get up 
and answer the telephone. The user can resume watching the 

10 show after interruption as if the program were on video 

tape. In addition, while watching a sports event the user 
can rewind back to a controversial pass, replay the play in 
slow motion, stop at the exact instance when the catch was 
made and zoom in on the receiver. Also, the user can apply 

15 an image sharpening filter to see if both of the receiver's 
feet touched the ground before the receiver went out of 
bounds . 

Timeshif ting by substantially simultaneously recording 
and playing back a data stream enables the following type 

20 of benefits/features for an end-user. While watching the 
season premiere of a television show, the viewer can 
literally "pause" the program in order to get up and answer 
the phone or get a snack from the refrigerator. After the 
interruption, the viewer can resume watching again without 

25 having missed anything. If the viewer came home 15 minutes 
late for the show but had started recording the program 
from the beginning, the viewer can begin watching right 
away instead of waiting for the show to be over and then 
rewinding the cassette tape. Also the viewer can replay 

30 scenes during a "live" broadcast (e.g. season premiere of a 
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show or sporting event) in case the viewer missed a line or 
an exciting play. In addition, while watching a sports 
event the user can rewind back to a controversial play, 
replay the play in slow motion, and stop at the exact: 
5 instance when the catch was made. To get a better view, 

the viewer can zoom in on a portion of the screen and apply 
image sharpening filters to magnify one part of the screen. 

An embodiment of a video transmission system 1300, 
shown in Figure 18, includes a broadcast encoder 13 04 which 
10 transmits to a plurality of receivers 1308, only one of 
which is shown in Figure 18. The broadcast encoder may 
multiplex videc transmission content 1802 which may be, for 
example, conventional television programming together with 
control information 1306. For example, the broadcast: 
15 encoder 1304 may transmit video information with television 
content from the content input 1802 with control 
information 1806 which may be provided, for example, over 
the vertical blanking interval (VBI) in a conventional 
airwave television broadcast. The control information 1306 
20 can also be included in a variety of other ways in 
connection with other video transmission transports 
including those transmissions transmitted over satellite 
systems, cable systems and computer networks. 

In one embodiment the control information may include 
25 a cue which signals a receiver 1808 that a particular 

portion of a video transmission is about to occur or is 
occurring. The receiver 1808 may be, for example, a 
computer with a video card, a television, a cable converter 
box, a satellite receiver or even a separate unit. 
30 Referring to Figure 19, an example of an embodiment of 

a screen display 1900 in accordance with the invention 
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which may be displayed on the receiver 1808 includes a 
portion 1902 which displays an ongoing video cransmissicn 
such as a television program. The television program 
displayed in portion 1902 may be an ongoing display of an 
5 ongoing television video transmission. A plurality of 

replay portions 1904 may be displayed in other areas of the 
screen as illustrated. 

Thus, upon selection of a desired function, the screen 
1900 may display the ongoing television program either live 

10 (or as recorded) together with available replay portions 
that have been separately recorded and may be scored in 
distinct files in association with the television program 
1902. The user then may select any of the sections 1904 
for display during or after the television video 

15 transmission displayed in the portion 1902 . During zhe 

course of the video transmission, the screen may include a 
full size display of the ongoing video transmission. Then, 
when the user desires to see what replay options are 
available, the screen 1900 may be selected, causing the 

20 ongoing video transmission to be displayed in a reduced 
format size in portion 1902. 

Referring to Figure 20, the broadcast encoder 1304, 
which may be a conventional computer server, may, in one 
embodiment, include software 2000 stored in an appropriate 

25 storage medium for implementing a system wherein the 
receiver acquires cues when desired content is being 
transmitted (block 2002) . For example, information chat 
would otherwise be displayed as a replay during a 
television sports broadcast can be accompanied by a 

30 distinct cue which may be detected by the receiver. When 
the receiver detects the cue, the receiver automatically 



25 



stores the ensuing video transmission as a dis-inct replay' 
which can be viewed by the receiver using the display 
screen format shown in Figure 19 for example. 

The cue may be developed, for example, in response to 
the detection of a characteristic such as the ncrmal 
transition from a normal video broadcast to a replay in a 
sports broadcast. A scene change detector may detect the 
fade in and fade out which often accompanies the transition 
to the replay. Alternatively, the production personnel at 
the content provides a digital signal encoded with the 
control information 1306 with content that should be stored 
as a replay is about to begin. In this way, -he user of 
the receiver 1808 may be provided with separate files of 
the various replays which accompany a video transmission, 
such as a television program. 

Automatically storing transmission segments may also 
be useful in connection with a variety of other video 
transmissions. For example a user may wish to store 
portions of the video transmission, such as favorite scenes 
from a movie, separately from the rest of the transmission. 
These stored portions may then be readily accessible 
independently of the rest of the video transmission for 
example using the display screen 1900. 

The encoder 1804 may encode the start cue in response 
to the receipt of cue information as described previously, 
as indicated at block 2004 in Figure 20. This start cue 
may be encoded in the control information IS 0 6 that sends 
appropriate content to the broadcast encoder 1804. The 
encoder 1304 then includes the digital information about 
the replay or cue with the rest of the transmission, either 
as a signal encoded with the actual broadcast or as a 



separate packet or region. For example, the information 
may be associated with closed caption scripting or may be 
included in a vertical blanking interval (VBI; . 

The content from the content source 1302, such as 
5 television content, may be transmitted together with the 
cue which may be part of the control information 1306, as 
indicated in block 2006. When the desired information has 
been transmitted, an end cue (block 2008) may be determined 
as described above in connection with the start cue. For 
10 example, a unique signature may be provided in content 

stream either in a separate packet or area associated with 
the television program or included with the television 
content information. The end cue is then transmitted 
(block 2010) with the content 1302 and control information 
15 1806 to the receiver 1308. 

At the receiver 1308, the start cue may be detected 
using the software 2100, as indicated at block 2102 in 
Figure 21. For example, where a distinct cue is sent, it 
may be detected in the video transmission itself or in a 
20 vertical blanking interval or in association with closed 
caption script as examples. Once it is detected, the 
receiver 1308 may activate the record function described 
previously, as indicated in block 2104. Thus, the user of 
the receiver 1308 may view the ongoing video transmission 
25 at the same time the selected replay portion is actively 
recorded. Next, the receiver 1308 may detect the end cue 
transmitted by the broadcast encoder 1804, as indicated in 
block 2106. In response to the detection of the end cue, 
the recording may be terminated as indicated at block 2108. 
30 In' some systems, the recording may actually continue but 
the file associated with a particular characteristic or 
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replay is closed. Thus, a distinct file associated with 
that characteristic or replay may be- separately stored and 
accessed even if recording continues with other material. 

At diamond 2110, a query determines whether a request 
for display of display options, such as the one shown in 
Figure 19 has been called. It may be called, for example, 
using computer mouse inputs or using a remote control as 
described previously. If so, the information displayed in 
Figure 19 may be displayed as indicated in block 2112. The 
user can then select either the viewing of the television 
program or any of the portions represented by the blocks 
1904 . 

In addition, ail of the replay clips may be 
automatically concatenated by the receiver fcr display one 
after another. This provides a "highlight package" that 
can be viewed to review highlights of the transmitted 
content. Also the catchup feature may be used with the 
replay. The transmitted content may be viewed in an 
accelerated fashion after viewing the replay to "catch up" 
to a live broadcast. 

Thus, a receiver may have a record of a video 
transmission together with recorded files which contain 
information about particular portions of that transmission 
which have been designated by the broadcaster as worthy of 
separate recording in separate files. For example, in a 
football broadcast, the replays may then be accessed by the 
user repeatedly in the course of the ongoing broadcast or 
thereafter as desired. This may be done by simply allowing 
the user to mouse click on the desired screen display 
portion 1904 associated with a particular replay. The 
screen display portions 1904 may each show one frame of the 



recorded information to assist the user in selecting the 
desired content for display. 

Thus, an embodiment of a method of substantially 
simultaneously recording and playing back a time -shifted 
5 video stream in accordance with the invention is disclosed. 
The specific arrangements and methods described herein are 
merely illustrative of the principles of this invention. 
For example, a similar embodiment may be used to store and 
retrieve other types of data streams besides video streams. 

10 Numerous modifications in form and detail may be made 
without departing from the scope of the described 
invention. Although this invention has been shown in 
relation to a particular embodiment, it should net oe 
considered so limited. Rather, the described invention is 

15 limited only by the scope of the appended claims. 
What is claimed is: 
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1 1. A method of displaying and recording a video 

2 transmission comprising: 

3 monitoring an ongoing video transmission for a 

4 predetermined characteristic; and 

5 upon detecting said characteristic, automatically 

6 recording said video transmission while said video 

7 transmission is being displayed. 

1 2. The method of claim 1 wherein monicoring an 

2 ongoing video transmission includes monitoring for the 

3 transition from a live broadcast to a replay. 

1 3. The method of claim 2 wherein monitoring an 

2 ongoing video transmission includes monitoring for a first 

3 cue to begin the recording of the video transmission and 

4 monitoring for a second cue to end the recording. 

1 4. The method of claim 1 further including 

2 displaying on a display screen the video transmission 

3 together with one or more representations of portions of 

4 the video transmission which have been recorded. 

1 5. The method of claim 1 wherein monitoring an 

2 ongoing video transmission for a predetermined 

3 characteristic includes monitoring the transmission for a 

4 digital signal indicating that recording should begin. 
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1 6. The method of claim 1 wherein monitoring an 

2 ongoing video transmission includes monitoring a closed 

3 caption transmission accompanying said video transmission. 

1 7. The method of claim 1 wherein monitoring an 

2 ongoing video transmission includes monitoring the vertical 

3 blanking interval . 

1 8. The method of claim 1 including storing said 

2 recorded video transmission information on a receiver that 

3 receives said video transmission. 

1 9. The method of claim 1 wherein monitoring an 

2 ongoing video transmission for a predetermined 

3 characteristic includes monitoring for a characteristic 

4 indicative of a replay. 

1 10. The method of claim 1 further including enabling 

2 the recorded video transmission to be replayed, and then 

3 accelerating the rate of replaying a portion of the 

4 transmission that occurred when the recorded video 

5 transmission was being played. 

1 11. A transmitter comprising: 

2 a monitor that monitors an ongoing video 

3 transmission and provides an indication when a 

4 characteristic is detected; and 

5 a transmission device that transmits said video 

6 transmission together with said indication. 
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1 12. The transmitter of claim 11 wherein said 

2 transmission device is a cable television transmitter . 

1 13. The transmitter of claim 11 wherein said 

2 transmission device is a satellite television transmitter. 

1 14. The transmitter of claim 11 wherein said 

2 transmission device is a over the air broadcast television 

3 transmitter. 

1 15. The transmitter of claim 14 wherein said 

2 transmission device is a transmitter that transmits said 

3 indication in the vertical blanking interval. 

1 16. An article comprising a medium for storing 

2 instructions that cause a computer to: 

3 monitor an ongoing video transmission for a 

4 ' predetermined characteristic; and 

5 upon detecting said characteristic, automatically 

6 record s^id video transmission while said video 

7 transmission is being displayed. 

1 17. The article of claim 16 further including 

2 instructions that cause a computer to monitor for the 

3 transition from a live broadcast to a replay. 

1 18. The article of claim 16 further including 

2 instructions that cause a computer to monitor for a first 
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3 cue to begin the recording of the video transmission and 

4 monitor for a second cue to end the recording. 

1 19. The article of claim 16 further including 

2 instructions that cause a computer to display on a display 

3 screen the video transmission together with one or more 

4 representations of portions of the video transmission which 

5 have been recorded. 

1 20. The article of claim 15 further including 

2 instructions that cause a compucer to monitor che 

3 transmission for a digital signal indicating that recording 

4 should begin. 

1 21. The article of claim 16 further including 

2 instructions that cause a computer to monitor for a 

3 characteristic indicative of a event to be recorded. 

1 22. A video receiver comprising: 

2 a firsc device to receive a video signal; 

3 a second device coupled to said first device to 

4 detect a characteristic of said signal; and 

5 a video recorder that records said video signal 

6 in response to the detection of said characteristic. 

1 23. The receiver of claim 22 wherein said second 

2 device detects a cue encoded with the video signal. 
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1 24. The receiver of claim 22 wherein said receiver is 

2 a computer system. 

1 25. The receiver of claim 22 wherein said receiver 

2 automatically concatenates a series of recorded sessions. 

1 26. The receiver of claim 22 wherein said second 

2 device detects a signal indicating that recording should 

3 start and another signal indicating that recording should 

4 stop. 

1 27. The receiver of claim 22 wherein said first 

2 device receives said video signal from a broadcast source, 

3 said signal including a cue to indicate the scart of 

4 recording by said video recorder. 
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TIME SHIFTING BY SIMULTANEOUSLY RECORDING 
AND PLAYING A DATA STREAM 

Abstract of the Disclosure 
A video stream may include portions that are stored 
while other portions are being displayed. In one 
embodiment, a portion of the video stream may be stored in 
5 digital storage media at one instance while in the next 

instance another portion of the stream is being read out of 
the storage media. 
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